Study Notes on Deep Learning III — Numerical Computation

This article mainly explores some key concepts in the fields of deep learning and optimization, including gradient, partial derivative, constrained optimization, and the KKT method. Below is the organization and summary of these contents: ### 1. Gradient and Partial Derivative - **Univariate Function**: For a univariate function \( f(x) \), the stationary point (extreme point) can be found by solving its derivative \( df/dx = 0 \). - **Multivariate Function**: - **Partial Derivative**: For a function with multiple inputs \( z = f(x, y) \), partial derivatives can be computed by differentiating with respect to each input separately.